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Abstract — We design iterative receiver schemes for a generic 
wireless communication system by treating channel estimation 
and information decoding as an inference problem in graphical 
models. We introduce a recently proposed inference framework 
that combines belief propagation (BP) and the mean field (MF) 
approximation and includes these algorithms as special cases. We 
also show that the expectation propagation and expectation max- 
imization algorithms can be embedded in the BP-MF framework 
with slight modifications. By applying the considered inference 
algorithms to our probabilistic model, we derive four different 
message-passing receiver schemes. Our numerical evaluation 
demonstrates that the receiver based on the BP-MF framework 
and its variant based on BP-EM yield the best compromise 
between performance, computational complexity and numerical 
stability among all candidate algorithms. 

I. Introduction 

The design of advanced receiver algorithms is crucial to 
meet the stringent requirements of modern communication 
systems. In this context, variational techniques performing 
Bayesian inference on probabilistic models HI have proven 
to be a very useful tool to design such algorithms. 

Among the variational techniques, belief propagation 
(BP) 121, El has found the most widespread use in digital 
communications. Originally applied to the decoding of chan- 
nel codes, BP has been shown to be especially efficient in 
discrete probabilistic models. An alternative to BP is the mean 
field (MF) approximation and its message-passing counter- 
part, usually referred to as variational message-passing 0. 
MF inference has been successfully applied to continuous 
probabilistic models involving probability density functions 
(pdfs) belonging to an exponential family, in which BP suf- 
fers from numerical intractability. Other notable examples of 
inference techniques are expectation-maximization (EM) 
and expectation propagation (EP) (6). EM is a special case 
of MF, where the approximate pdfs - referred to as be- 
liefs - are Dirac delta functions; EP can be seen as an 
approximation of BP where some beliefs are approximated 
by Gaussian pdfs. Some attempts have been made to find a 
unified framework encompassing all these techniques, as the 
a-divergence interpretation in Q and the region-based free 
energy approximations in 1 8 1 . Following the latter approach, a 
novel message-passing inference technique combining BP and 
the MF approximation was recently proposed in |9j. 

In this contribution, we investigate the design of receivers 
performing iterative channel estimation and data decoding in 
a generic wireless system. We review the combined inference 
technique in (9), which provides some degree of freedom in 



the choice of the parts of the factor graph in which either BP 
or MF is applied. Furthermore, we show that this framework 
can be modified to naturally embed EP, EM and BP with 
Gaussian approximation of some messages. We apply the 
mentioned inference techniques to the underlying probabilistic 
model of the wireless system and obtain four different re- 
ceiver algorithms. Our numerical assessment in a transmission 
scenario with a fast-varying channel shows that the receivers 
based on the combined BP-MF technique and its modified 
EM version are the most suitable among the investigated 
techniques, judging by the attained BER performance and the 
computational complexity. 

Notation: we denote by \I\ the cardinality of a finite set I; 
the relative complement of {i} in X is written as I \ i; the 
set {i E N | 1 < i < 7i } is denoted by [1 : n]. Boldface 
lowercase and uppercase letters are used to represent vectors 
and matrices, respectively; superscripts (-) T and (-) H denote 
transposition and Hermitian transposition, respectively. The 
Hadamard product of two vectors is denoted by 0. For a 
vector x = (atf | i e T) T , we write xj = (xj \ j E X\i) T ; for 
a matrix A e C mx ™, [A]^ denotes its (i,j)th entry, [A]" is 
the matrix A with the zth row and jth column deleted, [AJj a 
denotes the column vector ([A] fej - | k E [1 : m] \ i) T , and 
[A] i -■ is the row vector ([A] i k \ k G [1 : n] \ j). The pdf 
of a multivariate complex Gaussian distribution with mean 
fi and covariance matrix £ is denoted by CN(-;/x, S). We 
write f(x) oc g(x) when f(x) = cg(x) for some positive 
constant c. We denote by Q[-] the approximation of the pdf 
in the argument with a Gaussian pdf with the same mean and 
covariance matrix. The Dirac delta function is denoted by §(■). 

II. Message-Passing Inference Algorithms 

We begin by concisely describing the unified message- 
passing algorithm that combines the BP and MF approaches 
(refer to [9|). Then, we briefly show how other widespread 
inference algorithms can be obtained as particular instances 
or slight modifications of the unified framework. 

Let p(x) be an arbitrary pdf of a random vector x = 



(xi\i E T) which factorizes as 

P (x) = n f a (xa) = n / a ( Xa ) n / c ( Xc 



(i) 
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where x a is the vector of all variables Xi that are arguments 
of the function f a for all a E A. In the rightmost expression 
in (Q]) we have grouped the factors into two sets so that 



*4mf H Abp = and A M f U Abp = A. The factorization 
in can be visualized in a factor graph 0. We define 
Af(a) C I to be the set of indices of all variable nodes 
connected to f a in the factor graph; similarly, N~(i) C „4 
denotes the set of indices of all factor nodes connected to 
Xi. The parts of the graph that correspond to riae^BP / a ( x °) 
and to JlaG^MF / a ( Xa ) are referred to as "BP part" and "MF 
part", respectively. We denote the variable nodes in the BP 
part by Zbp = Uog^Ibp -^"( a ) anc ^ tne ones m tne MF by 

The combined BP-MF inference algorithm approximates the 
marginals p(xi) = J p(x)dxj, i £ I by auxiliary pdfs bi{xi) 
called beliefs. They are computed as (9] 

b i {x i ) = z i J] m^(a:0 J] m^Oc) (2) 
ce^BpnAA(i) ce^t M FnA/'(i) 

with 

m^ i (x i ) = z a Yl dxjUj^aixj) / (x ), 

j<Z/S(a)\i 

V a e .Abp, i £ jV(o) 

n^ifo) = exp / dx J n J _ ) . (a;j)ln/ a (x ) , 

\ J jeJV(a)\t / 

V a e -4mf, i G AA(a) 

ceA B pnAf(i)\a ceAiFrW(i) 

V i e Af(a),a £ A 

(3) 

where m and z a are constants that ensure normalized beliefs. 

Belief propagation is obtained as a particular case of BP- 
MF by setting Amf = 0, since in this case the expressions in 
(O reduce to the BP message computations. Similarly, mean 
field is an instance of BP-MF when Abp = 0- 

Expectation propagation is very similar to BP, the main 
difference being that it constrains the beliefs of some variables 
to be members of the exponential family. Such approximations 
can also be integrated in the BP-MF framework by modifying 
the messages 

m™_n(xi) oc -r^S [rii^aix^m^^Xi)] (4) 

for all i € 2ep Q 2bp, a £ A/"(i) PI Abp, where we assumed 
Gaussian approximations. 

The expectation-maximization algorithm was shown in (9) 
to be a special case of MF when the beliefs of some variables 
are constrained to be Dirac delta functions. Again, we include 
this approximation in the BP-MF framework. This leads to 
Hi-tai^i) = 5(Xi-Xi) for all i £ 2 EM C 2 MP and a £ A M f 
where Xi maximizes the unconstrained belief We refer to 
this modified algorithm as BP-EM. 

III. Probabilistic System Model 

In this section, we present the signal model of our inference 
problem and its graphical representation. These will establish 



the baseline for the derivation of message-passing receiver 
schemes. 

We analyze a system consisting of one transmitter and 
one receiver. A message represented by a vector u = 
(itfc | k £ [1 : K] ) T £ {0, 1} K of information bits is conveyed 
by sending N data and M pilot channel symbols having the 
sets of indices V C [1 : M + N] and V C [1 : M + N], 
respectively, such that V U V = [1 : M + N] and V n V = 0. 
Specifically, vector u is encoded and interleaved using a 
rate R = K/ (NL) channel code and a random interleaver 
into the vector c = (c£ | c„ £ {0,l} L ,n £ [1 : N]) T with 
length NL, For each n £ [1 : N], the subvector c„ = 
(cn \ ■ ■ ■ ,cl L ' ) ) T with length L is mapped to a data symbol 
Xi n € >Sd with i n £ T>, where So is a discrete modulation 
alphabet of size 2 L . The data symbols xq = (xi \ i £ T>) T 
are multiplexed with pilot symbols xp = (xj \ j £ V) T , 
which are randomly selected from the discrete modulation 
alphabet Sp. Finally, the aggregate vector of channel symbols 
x = (xi | i £ V U V) T is sent through a channel with the 
following input-output relationship: 

y = h©x + z. (5) 

The vector y = (j/j | i £ [1 : M + N}) T contains 
the observed signal samples at the receiver, h = 
(hi \ i £ [1 : M + N}) T is the vector of channel coefficients, 
and z = (zi \ i £ [1 : M + N}) 1 contains the samples of 
additive noise with pdf p(z) = CN (z; 0, 7 _1 lAf+jv) for some 
positive component precision 7. Note that (0 is able to model 
any channel with a multiplicative effect that is not affected by 
inter-symbol interference, e.g., a time-varying frequency-flat 
channel or the equivalent channel in the frequency domain in 
a multicarrier system. 

Based on the above signal model, we can state the prob- 
abilistic model which captures the dependencies between the 
system variables. The pdf of the collection of observed and 
unknown variables factorizes as 

p(y,h,Xb,c,u) 

= n ■fo ( h ^ x i) n ■frj (mm* 1 ) n ^ > c «) 

ieV j£T ne[l:N] 

/c(c,u) H f Uk (u k ) (6) 

ke[l:K] 

where f Di (h i7 Xi) = p(yi\hi,Xi) and fv^hj) = p{yj\hj) 
include observed values in y and are given by 

fDi(hi,Xi) = CN (hiX^yi,^ 1 ) , yi£V, (7) 
f P] (h J )=CN(h J x J ;y J , 1 - 1 ), Vj£V, (8) 

/n(h) = p{h) is the prior pdf of the vector of channel 
coefficients for which we set 

/ H (h)=CN(h;// h ,£ p h ), (9) 

/m„ (%i n j c n ) = p (xi n \c n ) characterizes the modulation map- 
ping, /c( c ; u ) — p( c l u ) accounts for the concatenation of the 
coding and interleaving operations and fu k (ut) = p{uk) is the 
prior uniform pmf of the fcth information bit. To obtain ©, 




Fig. 1. Factor graph representation of the pdf factorization in {6) with 
il, . . . , ijv £ P and j 6 V. 

we used the fact that y is independent on c and u conditioned 
on xd, h is independent on xp, c and u, the noise samples 
Zi are i.i.d., and each data symbol x in is independent on all 
other symbols conditioned on c„. The factorization in (|6]l can 
be visualized in the factor graph [ 1 1 depicted in Fig.Q] Note 
that the code graph is not explicitly given, the code structure 
being captured by fc- 

IV. Message-Passing Receiver Schemes 

In this section, we derive iterative receiver schemes by 
applying different inference algorithms to the factor graph in 
Fig. [T] We begin by defining the index sets 

A = {fu k \ke[l:K]}U {f c } U {/ M „ \ n e [1 : N}} 
U {f ?j | j g V} U {/ Dl | i g V} U {fa}, 

1 = {hi i g [1 : M + N}} U {x t | i G V} 

U{c ( i\...,c<fi ) }U{ui,...,UK}. 

The receiver has to infer the beliefs of the information bits 
using the observed vector y and prior knowledge, i.e., the 
pilot symbols and their set of indices V, the noise precision 
7, the channel statistics in (O, the modulation mapping and 
the structure of the channel code and interleaver. 

We will show that the BP algorithm resulting from setting 
Amf = yields messages of an intractable complexity. 
Assume that by running BP in the part of the graph containing 
the modulation and code constraints we obtain the messages 



(10) 



with i„ G £>,Vri G [1 : N], where fii n (s) represent extrinsic 
information on symbol Xi n . These messages are further passed 
as n Xin ^f v (xi n ) = ^ x . (xi n ). Then, for each i G V 
compute the messages 



(11) 



f .->hi(hi) oc / f Di (hi,Xi)n Xi ^f Dt (xi)dXi 



DC 



sG<Sd 

while for all j G V set 



ViS* 1 



(12) 



Note that the messages in ( fTTT ) are proportional to mixtures of 
Gaussian pdfs with |5d| = 2 L components. Then, after setting 



t ^f H (hi) = wr f l ,^. hi {hi) for all i g V and n hi ^f H (hi) 



tn 



BP 

h 



yh .(hi) for all i g V, the messages from the channel 
prior pdf fu read 

™y H ^(h t )<x f f H (h) [] n^f^dhj. (13) 

Using (O, (fTTl i and i\2i , the messages in (fT3T i become mixtures 
of Gaussian pdfs with 2 L ( Ar ~ 1 ) components for i G 2? and 2 LAr 
components for i <E V. Clearly, the complexity of computing 
these messages is intractable and one has to make additional 
approximations. 

A. Algorithm based on BP combined with Gaussian approxi- 
mation 

Since the intractability of the messages occur due to the 
mixtures in (fTTT i, we approximate those messages as proposed 
in fin , i.e., for each i G V we set 



m^Jh^^g 

with 



CN(h i; 



El 
«<(*)]7j2' 

«es D 



a hi,o 



7|s| s 



(14) 



(15) 



lM/n,o| 



where we have defined the normalized amplitudes of the 
Gaussian mixtures «i(s) = /?i (s) / (k^ |s| 2 ) with Ki = 

Y< S £S D Pi( s )l\ s \ 2 such that L sG ,s D a i( s ) = !■ We also denote 
the mean and variances from ( TTZi i with /ifi j)0 and u\ , j G P, 

and we define the vector /x^ = (/i/^.o « G [1 : M + iV]) T and 
the matrix with entries [£hkj = CT h o if * = 3 an d zero 
otherwise, for all i, j g [1 : M + N]. 

Now, using (O and (TBI , the message ( fT3l l becomes 

^aW*) « / CN ( h ; s h) cn (h- ^, [sy ii? ) dh^ 

oc CN (/ii;Mh 4 , c ,o'^ )C ) (16) 

with 

Mfti , c = ^ + [S p h ] ii? ([S£] ?ii + [S£h j)" 1 ( M ° h? - <), 

(17) 

These messages are further passed as extrinsic values, i.e., 



(hi 



messages are then computed: 



.(hi). For i G V, the following 



/ d ^i,(i.)« / / D< (/ii,iEi)n/ l< _ > ./ D4 (/ii)d/ii 



7" 



exp 



7" 



After passing extrinsic messages n Xi _j./ M (x^) = 
mj* _^ x . (x in ) for all i n g X>, n G [1 " : iV], we 
apply the BP update rule to compute the probabilities of 
the coded and interleaved bits (which is equivalent to MAP 



demapping), followed by BP decoding to obtain the beliefs 
of the information bits. 

B. Algorithm based on expectation propagation 

We set _4 M f = and l E p = {hi \ i G V}. Using © 
to compute _^ hi (hi) leads to messages proportional to 

Gaussian pdfs; consequently, the EP rule for m E ^^ hi (hi) 
reduces to the BP rule and outputs a Gaussian pdf as in ( [Tol l, 
since the operator Q[-] is an identity operator for Gaussian 
arguments. 

Specifically, using and and then (fTTT i and dT6b . we 
have for each i e D 

bhi(hi)=g n ht ^ }Vt (hi)m E ^ hi (hi) = CN (hf, fi^, cr 2 ,.) 
where 



Assume now that we have obtained the messages 
n xi^f D . (xi) (their expression will be given later) so that we 
can compute 

m / D %/ii(^) ocexP (^J n xi^f D . (xi)\nf Di (hi,Xi)dx^J 
oc CN (hi]^i hi . ,(jl. ) 



where 



Phi 



E Ms) 

seSv 



a h^,cPh z ,c 



IViS 



hi,c 



+ 71* 



sESd 



with 



Ms) = 



with the definition \x Xi = J n Xi ^f D (xi) Xidxi and cr 2 . = 
J n xi^fD t {xi)\xi - fi Xi \ 2 d^i. 

The messages nhi^f H (hi) — mM^ _^ hi (hi) are sent to the 
BP part and therefore are extrinsic values. When computing 
m^?^ h .(hi) we get the same expression as in ( fl6] l with the 
parameters given by (1171 1. Unlike for the previous algorithms, 
the following messages are beliefs, i.e., APP values: 

nhi^k^hi) = z hi m^ h .(hi) mf^ h .(hi) 
2 =CN(/ ll ;/i /li ,a 2 ), ViEV 



with 



s,7 + 



<M 2 



Phi = (v h l + o^c) (vhloVM* + ^^cM/u.c) (19) 



(7, — O, + tT, 



E seSn A(s)CN ( 2 / l ;^ 1 ,cS,7- 1 + < c kl 2 ) 

and ^hi.c, ct 2 . c as in (flTT l, Using (0]l again, we obtain 

CN (hi;n hi ,al.) 



Then, for all i G 2? we compute 



m A>%x i ( a; i) K ex P (^J n, H ^f Di (h l )lnf Dt (h l ,x l )dh^j 



m f ^ h (hi) oc 



CN 



2 



CN 



with 



® hi 
Phi 



'hi,c> 



Phi, ,Ohi,o) 



(18) 



oc CN I 37 2 ■ tT 



+ kj 2 7« + ki 2 ); 

and for all n G [1 : iV] we pass the messages 
n.^ ->/ Mn (&j ra ) = "i^f ^ x . (xi n ) to the modulation and 
coding part of the graph as extrinsic values. After running 
BP, we obtain ( fTOb and then pass the following APP values 
back to the MF part, for all n G [1 : N]: 



Note that, unlike ( fT3T > in BP with Gaussian approximation, for 
i G T> the values of fi^ Q and a\ computed with ( fT8l depend 
on all /x^ i0 and cr 2 . with j G V, j ^ i through dTTb . The 
parameters of rrFP^^Qij) are updated using dTvT > but with 
fihi,o and cr 2 . computed as above. Note that all messages 
that depend on the channel coefficients need to be updated in 
a sequential manner. 

The rest of messages are computed as in Section IIV-AI 

C. Algorithm based on the combined BP-MF framework 

The factor graph is split into the MF and BP parts by 
setting „4 M f = {/d s | i G £>} and „4 B p = A \ Amy- Such a 
splitting yields tractable and simple messages, takes advantage 
of the fact that BP works well with hard constraints and best 
exploits the correlation between the channel coefficients for 
the graphical representation in Fig. QjO. 

1 Alternatively, the same level of exploitation of the correlation is obtained 
by representing the channel variables as a single vector variable h and 
"moving" factor node /h to the MF part |9). 



>f Di fan) = Z *i n m f, 



BP 



MF 



D. Algorithm based on BP-EM 

We now apply EM for channel estimation, so we constrain 
bhi (hi) from the previous scheme to be Dirac delta functions 
for all i G V. The resulting messages are the same as in the 
previous subsection, except for nh^fi,. (hi) = S(hi — fi^) 
with computed as in ( fT9l ). 

E. Scheduling of message computations 

All algorithms employ the same message-passing schedul- 
ing: they start by sending messages m/ P ^hj(hj) correspond- 
ing to pilots and by initializing m/ D (hi) oc CN (hf, 0, oo); 
messages (computed according to the corresponding algo- 
rithm) are passed on up to the information bit variables - this 
completes the first iteration; each following iteration consists 
in passing messages up to the channel prior factor node and 
back; messages are passed back and forth until a predefined 
number of iterations is reached. All algorithms end by taking 
hard decisions on the beliefs of the information bits. 



V. Simulation Results 

We consider an OFDM system with the parameters given in 
Table H] and we evaluate by means of Monte-Carlo simulations 
the bit error rate (BER) performance of the receiver algorithms 
derived in Section [TV] We employ as a reference an algorithm 
which has perfect channel state information (CSI), i.e., it has 
prior knowledge of the channel coefficients h. 

We encountered numerical problems with the EP-based 
scheme due to the instability of EP in general, so we used 
the heuristic approach [7 1 to damp the updates of the beliefs 
b} li with a step-size e = 0.5. Also, the EP-based scheme has 
higher computational complexity than the others due to its 
message definition - it requires multiplication of a Gaussian 
pdf with a mixture of Gaussian pdfs, the approximation Q[-] 
and division of Gaussian pdfs - and to the sequentially of the 
message updates for the channel coefficiento 

Results in terms of BER versus signal-to-noise ratio (SNR) 
are given in Fig. [2] while the convergence of the BER with 
the number of iterations is illustrated in Fig. [3] The receivers 
based on EP, combined BP-MF and BP-EM exhibit simi- 
lar performance. They significantly outperform the receiver 
employing BP with Gaussian approximation. Note that even 
with a high pilot spacing Ap « 2.5Wcoh the performance of 
the former algorithms is close to that of the receiver having 
perfect CSI. These three algorithms converge in about 10-12 
iterations, while BP with Gaussian approximation converges 
a little faster, but to a higher BER value. Other results 
not presented here show that for a higher pilot density the 
algorithms converge faster, as expected. 



TABLE I 

Parameters of the OFDM system 



Parameter 


Value 


Subcarrier spacing 


15 kHz 


Number of active subcarriers 


M + N = 300 


Number of evenly spaced pilot symbols 


M = 10 


Pilot spacing 


A P rs 500 kHz 


Modulation scheme for data symbols 


16 QAM (L = 4) 


Modulation scheme for pilot symbols 


QPSK 


Convolutional channel code 


R = 1/3, (133, 171, 165) s 


Multipath channel model 


3GPP ETU 


Coherence bandwidth of the channel 


W coh ~ 200 kHz 



VI. Conclusions 

We formulated the problem of joint channel estimation and 
decoding in a wireless system as inference in a graphical 
model. To solve the inference problem, we resorted to a 
recently proposed message-passing framework that unifies the 
BP and MF algorithms and includes them as particular in- 
stances. Additionally, we illustrated how the combined frame- 
work can encompass the EP and EM inference algorithms. 

Based on the inference techniques considered, we derived 
four receiver algorithms. Since BP is not suitable for the 
studied problem, as it leads to intractable messages, we applied 
its variant which employs Gaussian approximation of the 

2 For the other receiver schemes, it can be shown that the parameters of all 
messages wnr?_^ h (hi) with i £ T> U V can be computed jointly and with a 
lower complexity. 
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Fig. 2. BER vs. SNR performance of the receiver algorithms for a number of 
pilot symbols M = 10, corresponding to a high pilot spacing Ap « 2.5W co h. 
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Fig. 3. Convergence of the BER performance from Fig. [2] at SNR = 12 dB. 

computationally cumbersome messages instead. However, our 
results showed that it performs significantly worse than the 
other proposed schemes. Considering the BER results, the 
computational complexity and stability of these schemes, we 
conclude that the receiver based on the combined BP-MF 
framework and its EM variant are the most effective receiver 
algorithms. 
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